Skip to content

Integrating VLS option on tests #8917

Open
enaples wants to merge 8 commits intoElementsProject:masterfrom
enaples:vls-hsmd-test
Open

Integrating VLS option on tests #8917
enaples wants to merge 8 commits intoElementsProject:masterfrom
enaples:vls-hsmd-test

Conversation

@enaples
Copy link
Collaborator

@enaples enaples commented Feb 26, 2026

Important

26.04 FREEZE March 11th: Non-bugfix PRs not ready by this date will wait for 26.06.

RC1 is scheduled on March 23rd

The final release is scheduled for April 15th.

Checklist

Before submitting the PR, ensure the following tasks are completed. If an item is not applicable to your PR, please mark it as checked:

  • The changelog has been updated in the relevant commit(s) according to the guidelines.
  • Tests have been added or modified to reflect the changes.
  • Documentation has been reviewed and updated as needed.
  • Related issues have been listed and linked, including any that this PR closes.
  • Important All PRs must consider how to reverse any persistent changes for tools/lightning-downgrade

Refactoring fixtures to test VLS implementation

The goal is to integrate VLS testing into the CLN test suite in a way that is self-contained — no changes to the pyln-testing library — and flexible enough to test different VLS implementations at different stages (different repos or commits) against the current CLN version, taking inspiration from tools/reckless.

The core idea is that ValidatingLightningSignerD should be generic: driven by the repos variable and optionally a target commit, it clones, builds and manages any conforming signer implementation. The fixture layer then decides which nodes use it and which implementation to test (so far only this is supported).

@enaples
Copy link
Collaborator Author

enaples commented Feb 26, 2026

On tests/test_pay.py, the test_simple_vls

@pytest.mark.openchannel('v1')
@pytest.mark.openchannel('v2')
def test_vls_simple(node_factory):
    l1, l2 = node_factory.line_graph(2, opts={'use_vls': True})

    inv = l2.rpc.invoice(123000, 'test_vls_simple', 'description')['bolt11']
    details = l1.dev_pay(inv, dev_use_shadow=False)
    assert details['status'] == 'complete'
    assert details['amount_msat'] == Millisatoshi(123000)
    assert details['destination'] == l2.info['id']

because some error on vls-side. I report logs for convenience:

lightningd-2 2026-02-26T11:29:47.518Z INFO    lightningd: v25.12-389-gb62be79-modded
lightningd-2 2026-02-26T11:29:47.664Z DEBUG   lightningd: Opened log file -
lightningd-2 2026-02-26T11:29:47.664Z DEBUG   lightningd: Opened log file /tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/log
lightningd-2 2026-02-26T11:29:47.666Z DEBUG   plugin-manager: started(81552) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/autoclean
lightningd-1 2026-02-26T11:29:47.518Z INFO    lightningd: v25.12-389-gb62be79-modded
lightningd-1 2026-02-26T11:29:47.666Z DEBUG   lightningd: Opened log file -
lightningd-1 2026-02-26T11:29:47.666Z DEBUG   lightningd: Opened log file /tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/log
lightningd-2 2026-02-26T11:29:47.667Z DEBUG   plugin-manager: started(81553) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/chanbackup
lightningd-1 2026-02-26T11:29:47.667Z DEBUG   plugin-manager: started(81554) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/autoclean
lightningd-2 2026-02-26T11:29:47.668Z DEBUG   plugin-manager: started(81555) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/bcli
lightningd-1 2026-02-26T11:29:47.668Z DEBUG   plugin-manager: started(81556) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/chanbackup
lightningd-2 2026-02-26T11:29:47.668Z DEBUG   plugin-manager: started(81557) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/commando
lightningd-1 2026-02-26T11:29:47.669Z DEBUG   plugin-manager: started(81558) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/bcli
lightningd-2 2026-02-26T11:29:47.669Z DEBUG   plugin-manager: started(81559) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/funder
lightningd-1 2026-02-26T11:29:47.670Z DEBUG   plugin-manager: started(81560) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/commando
lightningd-2 2026-02-26T11:29:47.670Z DEBUG   plugin-manager: started(81561) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/topology
lightningd-1 2026-02-26T11:29:47.672Z DEBUG   plugin-manager: started(81562) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/funder
lightningd-2 2026-02-26T11:29:47.674Z DEBUG   plugin-manager: started(81563) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/exposesecret
lightningd-1 2026-02-26T11:29:47.674Z DEBUG   plugin-manager: started(81564) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/topology
lightningd-1 2026-02-26T11:29:47.675Z DEBUG   plugin-manager: started(81565) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/exposesecret
lightningd-2 2026-02-26T11:29:47.675Z DEBUG   plugin-manager: started(81566) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/keysend
lightningd-2 2026-02-26T11:29:47.676Z DEBUG   plugin-manager: started(81567) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/offers
lightningd-1 2026-02-26T11:29:47.676Z DEBUG   plugin-manager: started(81568) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/keysend
lightningd-1 2026-02-26T11:29:47.677Z DEBUG   plugin-manager: started(81570) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/offers
lightningd-2 2026-02-26T11:29:47.678Z DEBUG   plugin-manager: started(81569) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/pay
lightningd-1 2026-02-26T11:29:47.678Z DEBUG   plugin-manager: started(81571) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/pay
lightningd-1 2026-02-26T11:29:47.681Z DEBUG   plugin-manager: started(81573) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/recklessrpc
lightningd-2 2026-02-26T11:29:47.681Z DEBUG   plugin-manager: started(81572) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/recklessrpc
lightningd-1 2026-02-26T11:29:47.682Z DEBUG   plugin-manager: started(81574) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/recover
lightningd-2 2026-02-26T11:29:47.682Z DEBUG   plugin-manager: started(81575) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/recover
lightningd-2 2026-02-26T11:29:47.683Z DEBUG   plugin-manager: started(81577) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/txprepare
lightningd-1 2026-02-26T11:29:47.684Z DEBUG   plugin-manager: started(81576) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/txprepare
lightningd-2 2026-02-26T11:29:47.684Z DEBUG   plugin-manager: started(81578) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-renepay
lightningd-1 2026-02-26T11:29:47.685Z DEBUG   plugin-manager: started(81579) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-renepay
lightningd-2 2026-02-26T11:29:47.687Z DEBUG   plugin-manager: started(81580) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-xpay
lightningd-1 2026-02-26T11:29:47.687Z DEBUG   plugin-manager: started(81581) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-xpay
lightningd-2 2026-02-26T11:29:47.688Z DEBUG   plugin-manager: started(81582) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/spenderp
lightningd-1 2026-02-26T11:29:47.689Z DEBUG   plugin-manager: started(81583) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/spenderp
lightningd-2 2026-02-26T11:29:47.689Z DEBUG   plugin-manager: started(81584) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-askrene
lightningd-1 2026-02-26T11:29:47.690Z DEBUG   plugin-manager: started(81585) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-askrene
lightningd-1 2026-02-26T11:29:47.691Z DEBUG   plugin-manager: started(81587) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/sql
lightningd-2 2026-02-26T11:29:47.692Z DEBUG   plugin-manager: started(81586) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/sql
lightningd-2 2026-02-26T11:29:47.693Z DEBUG   plugin-manager: started(81589) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-grpc
lightningd-1 2026-02-26T11:29:47.693Z DEBUG   plugin-manager: started(81588) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-grpc
lightningd-1 2026-02-26T11:29:47.694Z DEBUG   plugin-manager: started(81591) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/clnrest
lightningd-2 2026-02-26T11:29:47.695Z DEBUG   plugin-manager: started(81590) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/clnrest
lightningd-1 2026-02-26T11:29:47.695Z DEBUG   plugin-manager: started(81592) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-lsps-client
lightningd-2 2026-02-26T11:29:47.697Z DEBUG   plugin-manager: started(81593) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-lsps-client
lightningd-1 2026-02-26T11:29:47.697Z DEBUG   plugin-manager: started(81594) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-lsps-service
lightningd-2 2026-02-26T11:29:47.699Z DEBUG   plugin-manager: started(81595) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-lsps-service
lightningd-1 2026-02-26T11:29:47.699Z DEBUG   plugin-manager: started(81596) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/wss-proxy
lightningd-2 2026-02-26T11:29:47.700Z DEBUG   plugin-manager: started(81597) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/wss-proxy
lightningd-1 2026-02-26T11:29:47.701Z DEBUG   plugin-manager: started(81598) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-bip353
lightningd-2 2026-02-26T11:29:47.703Z DEBUG   plugin-manager: started(81599) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/cln-bip353
lightningd-1 2026-02-26T11:29:47.706Z DEBUG   plugin-manager: started(81600) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/bookkeeper
lightningd-2 2026-02-26T11:29:47.706Z DEBUG   plugin-manager: started(81601) /Users/emanuele/cln-versions/vls-hsmd-test/plugins/bookkeeper
lightningd-2 2026-02-26T11:29:47.712Z TRACE   plugin-cln-lsps-client: Plugin logging initialized
lightningd-2 2026-02-26T11:29:47.713Z TRACE   plugin-cln-lsps-service: Plugin logging initialized
lightningd-1 2026-02-26T11:29:47.714Z TRACE   plugin-cln-lsps-client: Plugin logging initialized
lightningd-1 2026-02-26T11:29:47.714Z TRACE   plugin-cln-lsps-service: Plugin logging initialized
lightningd-1 2026-02-26T11:29:47.717Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
lightningd-1 2026-02-26T11:29:47.717Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
lightningd-1 2026-02-26T11:29:47.717Z DEBUG   lightningd: io_break: check_plugins_manifests
lightningd-2 2026-02-26T11:29:47.717Z UNUSUAL plugin-bookkeeper: topic 'utxo_deposit' is not a known notification topic
lightningd-2 2026-02-26T11:29:47.717Z UNUSUAL plugin-bookkeeper: topic 'utxo_spend' is not a known notification topic
lightningd-2 2026-02-26T11:29:47.717Z DEBUG   lightningd: io_break: check_plugins_manifests
lightningd-2 2026-02-26T11:29:47.718Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_channeld
lightningd-1 2026-02-26T11:29:47.718Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_channeld
lightningd-1 2026-02-26T11:29:47.732Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_closingd
lightningd-2 2026-02-26T11:29:47.732Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_closingd
lightningd-1 2026-02-26T11:29:47.738Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_connectd
lightningd-2 2026-02-26T11:29:47.738Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_connectd
lightningd-1 2026-02-26T11:29:47.745Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_gossipd
lightningd-2 2026-02-26T11:29:47.745Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_gossipd
lightningd-2 2026-02-26T11:29:47.752Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_gossip_compactd
lightningd-1 2026-02-26T11:29:47.752Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_gossip_compactd
lightningd-1 2026-02-26T11:29:47.758Z DEBUG   lightningd: testing /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/target/debug/remote_hsmd_socket
lightningd-2 2026-02-26T11:29:47.758Z DEBUG   lightningd: testing /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/target/debug/remote_hsmd_socket
lightningd-1 2026-02-26T11:29:48.428Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_onchaind
lightningd-2 2026-02-26T11:29:49.170Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_onchaind
lightningd-1 2026-02-26T11:29:49.485Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_openingd
lightningd-2 2026-02-26T11:29:49.485Z DEBUG   lightningd: testing /Users/emanuele/cln-versions/vls-hsmd-test/lightningd/lightning_openingd
lightningd-1 2026-02-26T11:29:49.851Z DEBUG   hsmd: pid 81637, msgfd 70
lightningd-2 2026-02-26T11:29:49.851Z DEBUG   hsmd: pid 81638, msgfd 70
2026-02-26T11:29:49.861085Z  INFO vls-proxy/src/socket_main.rs:61: remote_hsmd_socket git_desc=v0.14.0-rc.2-38-gc54d3e3e starting
2026-02-26T11:29:49.861360Z  INFO vls-proxy/src/socket_main.rs:61: remote_hsmd_socket git_desc=v0.14.0-rc.2-38-gc54d3e3e starting
2026-02-26T11:29:49.862272Z  INFO vls-frontend/src/frontend.rs:161: frontend started
2026-02-26T11:29:49.862460Z  INFO vls-proxy/src/socket_main.rs:131: starting gRPC service on 127.0.0.1:64854
2026-02-26T11:29:49.862505Z  INFO vls-frontend/src/frontend.rs:161: frontend started
2026-02-26T11:29:49.862557Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:207: read loop 81637/3/0: start
2026-02-26T11:29:49.862642Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:215: read loop 81637/3/0 waiting for shutdown
2026-02-26T11:29:49.862687Z  INFO vls-proxy/src/socket_main.rs:131: starting gRPC service on 127.0.0.1:64850
2026-02-26T11:29:49.862854Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:207: read loop 81638/3/0: start
2026-02-26T11:29:49.862951Z  INFO vls-proxy/src/portfront.rs:91: waiting for signer_port to be ready
2026-02-26T11:29:49.862988Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:215: read loop 81638/3/0 waiting for shutdown
2026-02-26T11:29:49.863115Z  INFO vls-proxy/src/portfront.rs:91: waiting for signer_port to be ready
2026-02-26T11:29:49.863397Z ERROR /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:229: read loop 81637/3/0 saw error Protocol(TrailingBytes(9, 99)); ending
2026-02-26T11:29:49.863429Z  WARN /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:232: read loop 81637/3/0 terminated; triggering shutdown
lightningd: 2026-02-26T11:29:49.863517Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-1/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:218: read loop 81637/3/0 saw shutdown, calling exit
HSM sent unknown message type
2026-02-26T11:29:49.863637Z ERROR /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:229: read loop 81638/3/0 saw error Protocol(TrailingBytes(9, 99)); ending
2026-02-26T11:29:49.863667Z  WARN /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:232: read loop 81638/3/0 terminated; triggering shutdown
lightningd: 2026-02-26T11:29:49.863709Z  INFO /private/tmp/ltests-1uy95ld7/test_vls_simple_1/lightning-2/validating-lightning-signer/vls-proxy/src/grpc/signer_loop.rs:218: read loop 81638/3/0 saw shutdown, calling exit
HSM sent unknown message type
Time-out: can't find [re.compile('Server started with public key')] in logs
Time-out: can't find [re.compile('Server started with public key')] in logs
{'run_id': 285372585507352576, 'github_repository': 'ElementsProject/lightning', 'github_sha': 'b62be7967d323682b1086fb7876ca4ab7196a318', 'github_ref': None, 'github_ref_name': 'vls-hsmd-test', 'github_run_id': None, 'github_head_ref': None, 'github_run_number': None, 'github_base_ref': None, 'github_run_attempt': None, 'testname': 'test_vls_simple', 'start_time': 1772105332, 'end_time': 1772105447, 'outcome': 'fail'}
FAILEDLeaving base_dir /tmp/ltests-1uy95ld7 intact, it still has test sub-directories with failure details: ['test_vls_simple_1']

tests/test_pay.py::test_vls_simple ERROR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants